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Abstract 

In this paper, we propose a SLaT (Smoothing, Lifting and Thresholding) method with three 
stages for multiphase segmentation of color images corrupted by different degradations: noise, 
information loss, and blur. At the first stage, a convex variant of the Mumford-Shah model is 
applied to each channel to obtain a smooth image. We show that the model has unique solution 
under the different degradations. In order to properly handle the color information, the second stage 
is dimension lifting where we consider a new vector-valued image composed of the restored image 
and its transform in the secondary color space with additional information. This ensures that even 
if the first color space has highly correlated channels, we can still have enough information to give 
good segmentation results. In the last stage, we apply multichannel thresholding to the combined 
vector-valued image to find the segmentation. The number of phases is only required in the last 
stage, so users can choose or change it all without the need of solving the previous stages again. 
Experiments demonstrate that our SLaT method gives excellent results in terms of segmentation 
quality and CPU time in comparison with other state-of-the-art segmentation methods. 

Index Terms 

Mumford-Shah model, convex variational models, multiphase color image segmentation, color 
spaces. 


1. Introduction 

I MAGE segmentation is a fundamental and challenging task in image processing and computer 
vision. It can serve as a preliminary step for object recognition and interpretation, to mention a 
few. The goal of image segmentation is to group parts of the given image with similar characteristics 
together. These characteristics include, for example, edges, intensities, colors, and textures. For a 
human observer, image segmentation seems obvious, but consensus between different observers is 
seldom found. The problem is much more difficult to be solved by a computer. A nice overview of 
region-based and edge-based segmentation methods is given in fTSl l. In our work we investigate the 
image segmentation problem for color images corrupted by different types of degradations: noise, 
information loss and blur. 
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with (i > 1 be a given 
(i = 3 for the usual RGB 


Let C be a bounded open connected set, and f : Q ^ I 
vector-valued image. For example, = 1 for gray-scale images and 
(red-green-blue) color images. One has > 3 in many cases such as in hyperspectral imaging p8 | 
or in medical imaging p3| . In this paper, we are mainly concerned with color images (i.e. = 3) 

though our approach can be extended to higher-dimensional vector-valued images. Without loss of 
generality, we restrict the range of / to [0,1]^ and hence / G 

In literature, various studies have been carried out and many techniques have been considered 
for image segmentation 0, (g, 1^, 1^, l|4^-||45|, 1^ . For gray-scale images, i.e. = 1, 

Mumford and Shah proposed in (Ml, (Ml an energy minimization problem for image segmentation 
which finds optimal piecewise smooth approximations. More precisely, this problem was formulated 
in |[35l as 


Eusig,'^) ■= 2 


- gfdx +1 


/ 

Jn\v 


\Vg\^dx + Length(r), 


( 1 ) 


where A and fi are positive parameters, and g : ^ R is continuous in \ F but may be 

discontinuous across the sought-after boundary F. Here, the length of F can be written as 'H^(r), 
the 1 -dimensional Hausdorff measure in R?. Model ([T]) has attractive properties even though finding 
a globally optimal solution remains an open problem and it is an active area of research. A recent 
overview can be found in 

For image segmentation, the Chan-Vese model GD pioneered a simplification of functional Q 
where F partitions the image domain into two constant segments and thus \/g = 0 on Q\r. 
More generally, for K constant regions Qi, i G K}, the multiphase piecewise constant 

Mumford-Shah model (^ reads as 


\ ^ r 1 ^ 

^PCMS ({fii, Ci}f=i) = 2 + 2 

i=l i=l 


( 2 ) 


where Per(fi^) is the perimeter of Qi in Q, all Q^s are pairwise disjoint and Q = Uz=i The 
Chan-Vese model where = 2 in Q has many applications for two-phase image segmentation. 
Model Q is a nonconvex problem, so the obtained solutions are in general local minimizers. To 
overcome the problem, convex relaxation approaches (^, | [TT| , | [T3| , (^, graph cut method (25 1 
and fuzzy membership functions were proposed. 

After (TSl , many approaches decompose the segmentation process into several steps and here 
we give a brief overview of recent work in this direction. The paper (^ performs a simultaneous 
segmentation of the input image into arbitrarily many pieces using a modified version of model 
([T]) and the final segmented image results from a stopping rule using a multigrid approach. In | [T7| , 
an initial hierarchy of regions is obtained by greedy iterative region merging using model Q; the 
final segmentation is obtained by thresholding this hierarchy using hypothesis testing. The paper 
(^ first determines homogeneous regions in the noisy image with a special emphasis on topological 
changes; then each region is restored using model ([T]). Further multistage methods extending model 
^ can be found in (471 1 where wavelet frames were used, and in which is based on iterative 
thresholding of the minimizer of the ROF functional | |42| , just to cite a few. In the discrete setting, 
the piecewise constant Mumford-Shah model Q amounts to the classical Potts model (^. The 
use of this kind of functionals for image segmentation was pioneered by Geman and Geman in 
| |22| . In (441 , a coupled Potts model is used for direct partitioning of images using a convergent 
minimization scheme. 

In (^, a conceptually different two-stage method for the segmentation of gray-scale images 
was proposed. In the first stage, a smoothed solution g is extracted from the given image / by 
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minimizing a non-tight convexification of the Mumford-Shah model Q. The segmented image is 
obtained in the second stage by applying a thresholding technique to g. This approach was extended 
in fT^ to images corrupted by Poisson and Gamma noises. Since the basic concept of our method 
in this paper is similar, we will give more details on ||^, in Section]^ 

Extending or conceiving segmentation methods for color images is not a simple task since one 
needs to discriminate segments with respect to both luminance and chrominance information. The 
two-phase Chan-Vese model GD was generalized to deal with vector-valued images in [ |T4| by 
combining the information in the different channels using the data fidelity term. Many methods are 
applied in the usual RGB color space Q, [ |T4| , [ [T^ , [ [27| |, [ |2^ , [ |44| |, among others. It is often 

mentioned that the RGB color space is not well adapted to segmentation because for real-world 
images the R, G and B channels can be highly correlated. In [ [4T| , RGB images are transformed 
into HSI (hue, saturation, and intensity) color space in order to perform segmentation. In Q a 
general segmentation approach is developed for gray-value images and further extended to color 
images in the RGB, the HSV (hue, saturation, and value) and the CB (chromaticity-brightness) 
color spaces. However, a study on this point in [ [37| shows that the Lab (perceived lightness, red- 
green and yellow-blue) color space defined by the CIE (Commission Internationale de TEclairage) 
is better adapted for color image segmentation than the RGB and the HSI color spaces. In (Tfl 
RGB input images are first converted to Lab space. In color features are described using the 
Lab color space and texture using histograms in RGB space. 

A careful examination of the methods that transform a given RGB image to another color space 
(HSI, CB, Lab, ...) before performing the segmentation task shows that these algorithms are always 
applied only to noise-free RGB images (though these images unavoidably contain quantization and 
compression noise). Lor instance, this is the case of 0, (nl, ( 43 , among others. One of the 
main reasons is that if the input RGB image is degraded, the degradation would be hard to control 
after a transformation to another color space [ [T7| . 

Our goal is to develop an image segmentation method that has the following properties: 


(a) work on vector-valued (color) images possibly corrupted with noise, blur and missing data; 

(b) initialization independent and non-supervised (the number of segments is not fixed in advance); 

(c) avoid dealing with nonconvex and combinatorial methods for the sake of stability; 

(d) no need to solve the whole problem again when the number of segments required is changed; 

(e) take into account perceptual edges between colors and between intensities so as to detect 
vector-valued objects with edges and also objects without edges. 

Contributions. The main contribution of this paper is to propose a segmentation method having 
all these properties. Goals (a)-(d) lead us to explore possible extensions of the methods ||^, [ [T^ to 
vector-valued (color) images. Goal (e) requires finding a way to use information from perceptual 
color spaces even though our input images are corrupted; see goal (a). Let Vi and V 2 be two color 
spaces. Our method has the following 3 steps: 


1) Let the given degraded image be in Vi. The convex variational model is applied in 

parallel to each channel of Vi. This yields a restored smooth image. We show that the model 
has unique solution. 

2) The second stage consists of color dimension lifting: we transform the smooth color image 
obtained at Stage 1 to a secondary color space V 2 that provides us with complementary 
information. Then we combine these images as a new vector-valued image composed of all 
the channels from color spaces Vi and V 2 . 

3) According to the desired number of phases iT, we apply a multichannel thresholding to the 
combined V 1 -V 2 image to obtain a segmented image. 
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We call our method “SLaT” for Smoothing, Lifting and Thresholding. Unlike the methods that 
perform segmentation in a different color space like Q, [ [T7| , [ [3^ , [ |4T| , we can deal with 
degraded images thanks to Stage 1 which yields a smooth image that we can transform to another 
color space. We will fix Vi to be the RGB color space since one usually has RGB color images. 
We use the Lab color space ID as the secondary color space V 2 since it is often recommended 
for color segmentation fTf} , [ fTSl l, [ [37| . The crucial importance of the dimension lifting Stage 2 is 
illustrated in Fig. which shows the results without Stage 2, i.e. V 2 = 0 (middle) or with Stage 
2 (right). To the best of our knowledge, it is the first time that two color spaces are used jointly 
in variational methods for segmentation. This provides us with additional information on the color 



Fig. 1. Segmentation results for a noisy image (a) without the dimension lifting in Stage 2 (b) and with Stage 2 (c). 


image so that in all cases we can obtain very good segmentation results. The number of phases K 
is needed only in Stage 3. Its value can reasonably be selected based on the RGB image obtained 
at Stage 1. 

Extensive numerical tests on synthetic and real-world images have shown that our method 
outperforms state-of-the-art variational segmentation methods like pT| , [ |44| in terms of 

segmentation quality, speed and parallelism of the algorithm, and the ability to segment images 
corrupted by different kind of degradations. 

Outline. In Section]^ we briefly review the models in ||^, Our SLaT segmentation method 
is presented in Section m In Section |IV[ we provide experimental results on synthetic and real- 
world images. Conclusion remarks are given in Section |V| 


IT Review oe the Two-stage Segmentation Methods in [ fT^ 

The methods in for the segmentation of gray-scale images are motivated by the obser¬ 

vation that one can obtain a good segmentation by properly thresholding a smooth approximation 
of the given image. Thus in their first stage, these methods solve a minimization problem of the 
form 


inf 

geW^’^iQ) 




+ / iVgldx 


( 3 ) 


where $(/, g) is the data fidelity term, g and A are positive parameters. We note that the model ^ 
is a convex non-tight relaxation of the Mumford-Shah model in ([T]). Paper ^ considers $(/, ^) = 
(/ ~ where ^ is a given blurring operator; when / is degraded by Poisson or Gamma noise, 
the statistically justified choice $(/, g) = —/ log(^^) is used in Under a weak assumption, 

the functional in Q has a unique minimizer, say g, which is a smooth approximation of /. The 
second stage is to use the K-means algorithm to determine the thresholds for segmentation. 

These methods have important advantages: they can segment degraded images and the minimizer 
g is unique. Further, the segmentation stage being independent from the optimization problem Q, 
one can change the number of phases K without solving Q again. 
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III. SLaT: Our Segmentation Method eor Color Images 

Let / = (/i,..., /rf) be a given color image with channels /^ : ^ M, i = 1, • • • , rf. For / 

an RGB image, = 3. This given image / is typically a blurred and noisy version of an original 
unknown image. It can also be incomplete: we denote by fig the open nonempty subset of Q 
where the given fi is known for channel i. Our SLaT segmentation method consists of three stages 
described next. 


A. First Stage: Recovery of a Smooth Image 

First, we restore each channel fi of / separately by minimizing the functional E below 

^idi) = ^ 9 i)dx + [ \Vgydx+ [ \Vgi\dx, i = l,...,d, (4) 

^ JQ ^ Jci Jn 

where | • | stands for Euclidian norm and Uif) is the characteristic function of fig, i.e. 


cji{x) 


1, X G f^g, 

0 , 


( 5 ) 


For $ in Q we consider the following options: 

i) $(/, ^) = (/ — AgY, the usual choice; 

ii) ^{f^g) = Ag — f\og{Ag) if data are corrupted by Poisson noise. 


The restoration st ep am ounts to the first stage in |[^, in the simple case when d = 1 and 
fin = Theorem |lll. 1 below proves the existence and the uniqueness of the minimizer of Q. In 


view of Q and ([5), we define the linear operator (uiA) by 


{(jJiA) : u{x) G i-A (jJi{x){Au){x) G 


( 6 ) 


Theorem III.l. Let Ft be a bounded connected open subset ofM? with a Lipschitz boundary. Let 
A : Lp‘{Ft) Lp‘{Ft) be bounded and linear. For i G rf}, assume that fi G Lp‘{Ft) and 

that Ys^eriujiA) H Ker(V) = {0} where Ker stands for null-space. Then 0 with either = 

{fi - -A-gif or ^{fi,gi) = Agi - fi\og{Agi) has a unique minimizer gi G 


The proof is outlined in Appendix I. The condition Ker(6c;^Al) H Ker(V) = {0} is mild—it 
means that Kei^UiA) does not contain constant images. 

The discrete model. In the discrete setting. Ft is an array of pixels, say of size M x N, and our 
model Q reads as 

E{gi) = + ^\\'^9i\\F + llVpilb,!, i = l,...,d. 


Here 


'^{fi,9i) := E ~ •^9if)j or '^{fi,9i) ■= E G* ' “ fi^og{Agi))) .. 

The operator V = (V^;, Vy) is discretized using backward differences with Neumann boundary 
conditions. Further, || • ||^ is the Frobenius norm, so 


\\'^9^fF + {^y9i?j), 

jen 
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and IIII 2,1 is the usual discretization of the TV semi-norm given by 

||V^2||2,1 = {sxgi)‘^j + 

jen 

For each i, the minimizer gi can be computed easily using different methods, for example the 
primal-dual algorithm [ |T0||, [|T^ , alternating direction method with multipliers (ADMM) ||^, or the 
split-Bregman algorithm ]23[. Then we rescale each gi onto [0,1] to obtain {gi}i^i G [0,1]^. 

B. Second Stage: Dimension Lifting with Secondary Color Space 

For the ease of presentation, in the following, we assume Vi is the RGB color space. The goal 
in color segmentation is to recover segments both in the luminance and in the chromaticity of 
the image. It is well known that the R, G and B channels can be highly correlated. For instance, 
the R, G and B channels of the output of Stage 1 for the noisy pyramid image in Fig. are 
depicted in Fig. (a)-(c). One can hardly expect to make a meaningful segmentation based on 
these channels—see the result in Fig. (b), as well as Fig. where other contemporary methods 
are compared. 


(b) G channel g 2 (c) B channel ^3 


(e) a channel g^ (f) b channel g\ 

Fig. 2. Channels comparison for the restored (smoothed) g in Stage 1 used in Fig.H (a)-(c): the R, G and B channels 
of g\ (d)-(f): the L, a and b channels of f - the Lab transform of g. Both g and g were used to obtain the result in 



(a) R channel gi 



(d) L channel g\ 



Stage 1 provides us with a restored smooth image g. In Stage 2, we perform dimension lifting 
in order to acquire additional information on g from a different color space that will help the 
segmentation in Stage 3. The choice is delicate. Popular choices of less-correlated color spaces 
include HSV, HSI, CB and Lab, as described in the Introduction. The Lab color space was created 
by the GIF with the aim to be perceptually uniform ED in the sense that the numerical difference 
between two colors is proportional to perceived color difference. This is an important property 
for color image segmentation, see e.g. |T7), ||T§, [ [37| . For this reason in the following we use 
the Lab as the additional color space. Here the L channel correlates to perceived lightness, while 
the a and b channels correlate approximately with red-green and yellow-blue, respectively. As an 
example we show in Fig. (d)-(f) the L, a and b channels of the smooth g in Stage 1 for the noisy 
pyramid in Fig. [^(a). The result in Fig. [^(c) have shown that this additional color space helps the 
segmentation significantly. 
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Let g' denote Lab transform of g. In order to compare g' with g G [0,1]^, we rescale on [0,1] 
the channels of g' which yields an image denoted by g^ G [0,1]^. By stacking together g and g^ 
we obtain a new vector-valued image with 2d ^ Q channels: 

Our segmentation in Stage 3 is done on this 


C. Third Stage: Thresholding 

Given the vector-valued image G [0,1]^^ for = 3 from Stage 2 we want now to segment it 
into K segments. Here we design a properly adapted strategy to partition vector-valued images into 
K segments. It is based on the Jf-means algorithm because of its simplicity and good asymp¬ 
totic properties. According to the value of K, the algorithm clusters all points of {^*(x) : x G 12} 
into K Voronoi-shaped cells, say Si U112 * * = 12. Then we compute the mean vector Ck G 

on each cell by 


Ck = 


/sfc 

Is. 




( 7 ) 


We recall that each entry Ck[i] for i = 1, • • • , 6 is a value belonging to {R, G, B, L, a, b}, respec¬ 
tively. Using we separate into K phases by 


Uk'- eQ : \\g*{x) - Ck\ 


2 = min \\g*{x) 
l<j<K 


Cj\\2 


}, k = l,...,K. 


( 8 ) 


It is easy to verify that {12;^}^^ are disjoint and that U^i ^k = The use of the £2 distance 
here follows from our model Q as well as from the properties of the Lab color space [ |T7| , pT| . 
Note that in the simple case when c/^ G M (i.e., d = 1), the thresholding scheme ([^ reduces to the 
one used in 10, (H. 


D. The SLaT Algorithm 

We summarize our 3-stage segmentation method for color images in Algorithm Like the 
Mumford-Shah model, our model Q has two parameters A and /i. Extensive numerical tests have 
shown that we can fix /i = 1. We choose A empirically; the method is quite stable with respect to 
this choice. 


Algorithm 1: Three-stage Segmentation Method (SLaT) for Color Images 
Input: Given color image / G Vi and color space V 2 . 

Output: Phases 12/^, A; = 1,..., Jf. 

1 Stage one: compute gi the minimizer in Q, rescale it on [0,1] for i = 1,2,3 and set 
g = ( 51 , 52 , 53 ) in Vi; 

2 Stage two: 

3 compute G V 2 , the transform of ^ in V 2 , to obtain g^ = (^ 1,^25 ^ 3 )’ 

4 form g* = (51,52,53,51,5^53); 

5 Stage three: 

6 choose K, apply the Jf-means algorithm to obtain in Q and find the segments 

A: = 1,..., iT using (El. 


We emphasize that our method is quite suitable for parallelism since {gi)^^i in Stage 1 can be 
computed in parallel. 






IV. Experimental Results 


In this section, we compare our SLaT method with three state-of-the-art variational color seg¬ 
mentation methods pTf , p9t , Method HD uses fuzzy membership functions to approximate 
the piecewise constant Mumford-Shah model Q. Method uses a primal-dual algorithm to 


solve a convex relax model of ^ with a fixed code book. Method [ [44| uses an ADMM algorithm 
to solve the model Q (without phase number K) with Potts priors. The codes we used are provided 
by the authors, and the parameters in the codes were chosen by trial and error to give the best 
results of each method. For our model Q, we fix /i = 1 and only vary A. In the segmented figures 
below, each phase is represented by the average intensity of that phase in the image. All the results 
were tested on a MacBook with 2.4 GHz processor and 4GB RAM, and Matlab R2014a. 









(v) Pyramid 


Fig. 3. Images used in our tests. 


We present the tests on two synthesis and seven real-world images given in Fig.[^ The images are 
all in RGB color space. We considered combinations of three different forms of image degradation: 
noise, information loss, and blur. The Gaussian and Poisson noisy images are all generated using 
Matlab function imnoise. For Gaussian noisy images, the Gaussian noise we added are all of 
mean 0 and variance 0.001 or 0.1. To apply the Poisson noise, we linearly stretch the given image 
/ to [1, 255] first, then linearly stretch the noisy image back to [0, 1] for testing. The mean of the 
Poisson distribution is 10. For information loss case, we deleted 60% pixels values randomly. The 
blur in the test images were all obtained by a vertical motion-blur with 10 pixels length. 

In Stage 1 of our method, the primal-dual algorithm [ [T0| , and the split-Bregman algorithm 
II 23 I are adopted to solve (HI) for $(/, g) = Ag — f log(Ag) and $(/, g) — if — Ag^, respectively. 

^^ + -A 

We terminate the iterations when (fc+i)|| ^ < 10 for i = 1, 2,3 or when the maximum 

iteration number 200 is reached. In Stage 2, the transformation from RGB to Lab color spaces 
is implemented by Matlab build-in function makecform(' srgb21ab' ) . In Stage 3, given 
the user defined number of phases K, the thresholds are determined automatically by Matlab 
K-means function kmeans. Since is calculated prior to the choice of K, users can try different 
K and segment the image all without re-computing 
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A. Segmentation of Synthetic Images 

Example 1. Six-phase segmentation. Fig. gives the result on a six-phase synthetic image 
containing five overlapping circles with different colors. The image is corrupted by Gaussian noise, 
information loss, and blur, see Fig. (A), (B), and (C) respectively. From the figures, we see that 
method Li et al pT| and method Storath et al both fail for the three experiments while 
method Pock et al [ [3^ fails for the case of information lost. 

Table [T| shows the segmentation accuracy by giving the ratio of the number of correctly segmented 
pixels to the total number of pixels. The best ratios are printed in bold face. From the table, we see 
that our method gives the highest accuracy for the case of information loss and blur. For denoising, 
method is 0.02% better. Table gives the iteration numbers of each method and the CPU 
time cost. We see that our method outperforms the others compared. Moreover, if using parallel 
technique, the time can be reduced roughly by a factor of 3. 



(A) Noisy image 



(B) Information 
loss + noise 



(C) Blur + noise 




(Cl) Method (C2) Method 


(B3) Method 



(B4) Ours 


(C4) Ours 


(A4) Ours 





Fig. 4. Six-phase synthetic image segmentation (size: 100 x 100). (A): Given Gaussian noisy image with mean 0 and 
variance 0.1; (B): Given Gaussian noisy image with 60% information loss; (C): Given blurry image with Gaussian noise; 
(A1-A4), (B1-B4) and (C1-C4): Results of methods pT| , | [44) , and our SLaT on (A), (B) and (C), respectively. 


Example 2. Eour-phase segmentation. Our next test is on a four-phase synthetic image containing 
four rectangles with different colors, see Fig. The variable illumination in the figure make the 
segmentation very challenging. The results shows that in all cases (noise, information loss and blur) 
all three competing methods pT| , p9| , fail while our method gives extremely good results. 
Table [II| shows further that the time cost of our method is the least. 
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TABLE I 

Comparison of percentage of correct pixels for the 6-phase synthetic image. 



Method fTT] 

Method 1^ 

Method [44| 

Our SLaT method 

Fig.g 

(A) 

70.11% 

99.53% 

82.55% 

99.51% 

(B) 

13.90% 

16.92% 

85.04% 

99.25% 

(C) 

28.08% 

98.58% 

74.77% 

98.88% 

Average 

37.36% 

71.68% 

80.79% 

99.21% 


TABLE II 

Iteration numbers and CPU time in seconds. 



Method 1^ 

Method 1^ 

Method |44) 

Our SLaT method 

Fig. 

iter. 

time 

iter. 

time 

iter. 

time 

iter, for {gi}^^ 

time 



(A) 

200 

5.03 

150 

6.02 

20 

4.40 

(92, 86, 98) 

2.53 

(B) 

200 

5.65 

150 

4.01 

16 

3.65 

(98, 95, 106) 

2.73 

(C) 

200 

6.54 

150 

4.03 

17 

4.18 

(97, 95, 94) 

2.48 


(A) 

200 

13.92 

150 

13.89 

17 

16.89 

(54, 54, 51) 

5.47 


5 

(B) 

200 

13.16 

150 

14.32 

14 

13.62 

(101, 92, 88) 

7.74 


(C) 

200 

17.68 

150 

16.37 

16 

15.75 

(154, 147, 142) 

9.89 


1 

(A) 

200 

10.58 

150 

7.53 

19 

11.62 

(50, 73, 93) 

5.11 

(B) 

200 

9.59 

150 

7.36 

20 

14.64 

(84, 105, 115) 

6.43 

(C) 

200 

10.39 

150 

7.39 

19 

9.76 

(200, 200, 200) 

17.75 


(A) 

200 

44.26 

150 

66.01 

19 

106.35 

(97, 106, 109) 

25.13 


M 

(B) 

200 

52.12 

150 

54.76 

20 

110.68 

(148, 161, 171) 

38.30 


(C) 

200 

44.51 

150 

55.09 

18 

101.09 

(116, 125, 124) 

30.00 


(A) 

200 

17.76 

150 

19.02 

16 

25.08 

(80, 83, 99) 

20.99 


N 

(B) 

200 

18.41 

150 

16.45 

16 

28.33 

(109, 114, 129) 

22.45 


(C) 

200 

18.02 

150 

18.21 

15 

31.93 

(127, 120, 144) 

30.92 


1 

(A) 

200 

18.47 

150 

19.62 

15 

27.56 

(47, 42, 62) 

10.98 

(B) 

200 

17.35 

150 

16.63 

15 

26.63 

(86, 85, 93) 

15.93 

(C) 

200 

18.07 

150 

17.61 

15 

23.13 

(48, 48, 52) 

15.02 


(A) 

200 

24.57 

150 

31.64 

20 

56.28 

(101, 95, 94) 

27.29 
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(B) 

200 

27.15 

150 

28.92 

21 

63.02 

(154, 142, 131) 

27.89 


(C) 

200 

26.54 

150 

29.79 

20 

55.45 

(161, 147, 141) 

33.79 


(A) 

200 

26.62 

150 

32.87 

17 

87.13 

(35, 35, 36) 

14.23 
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(B) 

200 

24.77 

150 

26.39 

16 

60.98 

(102, 102, 103) 

18.99 


(C) 

200 

25.26 

150 

31.16 

18 

77.73 

(48, 50, 58) 

18.15 


(A) 

200 

32.23 

150 

41.91 

19 

47.12 

(106, 102, 108) 

21.93 
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(B) 

200 

34.83 

150 

44.70 

20 

53.48 

(116, 116, 117) 

23.95 


(C) 

200 

35.01 

150 

49.93 

19 

49.14 

(67, 65, 63) 

21.04 

Average 

200 

22.17 

150 

25.25 

18 

41.69 

(99, 99, 104) 

17.67 
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(B) Information (Bl) Method (B2) Method 

loss + noise '—^ '—' 


(B3) Method ^ 


(B4) Ours 



Fig. 5. Four-phase synthetic image segmentation (size: 256 x 256). (A): Given Gaussian noisy image with mean 0 and 
variance 0.001; (B): Given Gaussian noisy image with 60% information loss; (C): Given blurry image with Gaussian noise; 
(A1-A4), (B1-B4) and (C1-C4): Results of methods pT) , (3^, ||^, and our SLaT on (A), (B) and (C), respectively. 


B. Segmentation of Real-world Color Images 

In this section, we compare our method with the three competing methods for 7 real-world 
color images in two-phase and multiphase segmentations, see Figs. [6]-[T^ We see from the figures 
that our method is far superior than those by the competing methods. The timing of the methods 
given in Table [I^ shows that our method in most of the cases gives the least timing. Again, we 
emphasize that our method is easily parallelizable. All presented experiments clearly show that all 
goals listed in Introduction are fulfilled. 


V. Conclusions 

In this paper we proposed a three-stage image segmentation method for color images. At the 
first stage of our method, a convex variational model is used in parallel on each channel of the color 
image to obtain a smooth color image. Then in the second stage we transform this smooth image 
to a secondary color space so as to obtain additional information of the image in the less-correlated 
color space. In the last stage, multichannel thresholding is used to threshold the combined image 
from the two color spaces. The new three-stage method, named SLaT for Smoothing, Lifting and 
Thresholding, has the ability to segment images corrupted by noise, blur, or when some pixel 
information is lost. Experimental results on RGB images coupled with Lab secondary color space 
demonstrate that our method gives much better segmentation results than some state-of-the-art 
segmentation models both in terms of quality and CPU time cost. Our future work includes finding 
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(A) Noisy image 



(B) Information 
loss + noise 



(C) Blur + noise 





(B4) Ours 



(C4) Ours 


Fig. 6. Two-phase rose segmentation (size: 303 x 250). (A): Given Poisson noisy image; (B): Given Poisson noisy 
image with 60% information loss; (C): Given blurry image with Poisson noise; (A1-A4), (B1-B4) and (C1-C4): Results 
of methods 1^ , | [44) , and our SLaT on (A), (B) and (C), respectively. 


an automatical way to determine A and possibly an improved model Q that can better promote 
geometry. 
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Appendix I: Prooe of Theorem IIILII 

First consider ^{fi,gi) = (/* - Agi)^. Using E{gi) defined in (|^ can be rewritten as 

E{gi) ^ ^ [ {c^i-fi-{c^iA)gifdx+^ [ \Vgi\‘^dx+ [ \Vgi\dx. 

^ Jn ^ Jn Jn 

Noticing that Ui- fi G and that {uoiA) : Lp‘{^) is linear and bounded, the statement 

follows from Theorem 2.4.]. 

Next consider that = Agi — fi \og{Agi). Then 

Eigi) = ^ - K • fi)^og{Agi)dx + ^\\Vgi\\l2(^^) + ||V5fi||i2(Q). 
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(A) Noisy image 


(B) Information 
loss + noise 


(C) Blur + noise 





(C4) Ours 


Fig. 7. Four-phase sunflower segmentation (size: 375 x 500). (A): Given Gaussian noisy image with mean 0 and 
variance 0.1; (B): Given Gaussian noisy image with 60% information loss; (C): Given blurry image with Gaussian noise; 
(A1-A4), (B1-B4) and (C1-C4): Results of methods and our SLaT on (A), (B) and (C), respectively. 



(B) Information 
loss + noise 



(C) Blur + noise 





Fig. 8. Two-phase pyramid segmentation (size: 321 x 481). (A): Given Gaussian noisy image with mean 0 and variance 
0.001; (B): Given Gaussian noisy image with 60% information loss; (C): Given blurry image with Gaussian noise; 
(A1-A4), (B1-B4) and (C1-C4): Results of methods | [44) , and our SLaT on (A), (B) and (C), respectively. 


I) Existence: Since is a reflective Banach space and E{gi) is convex lower semi-continuous, 

by Proposition 1.2] we need to prove that E{gi) is coercive on i.e. that E{gi) +oo 

\\gi\\w^’‘^{Q) •= \\9i\\L^{Q) + \\'^9i\\L^{Q) +^* 
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(A) Noisy image 



(B) Information 
loss + noise 



(C) Blur + noise 




(B4) Ours 



(C4) Ours 


Fig. 9. Two-phase kangaroo segmentation (size: 321 x 481). (A): Given Poisson noisy image; (B): Given Poisson noisy 
image with 60% information loss; (C): Given blurry image with Poisson noise; (A1-A4), (B1-B4) and (C1-C4): Results 
of methods | [3T] , | [44) , and our SLaT on (A), (B) and (C), respectively. 


The function Agi ^ {Agi — f log Agi) is strictly convex with a minimizer pointwisely satisfying 
^9i = / ^ [0,1], hence ^{fi,gi) > 0. Thus \\Vgi\\L 2 (Q) is upper bounded by E{gi) > 0 for any 
gi G and / 0. Using the Poincare inequality, see we have: 

Wdi - ^ C^ill< CiE{gi), (9) 

where Ci > 0 is a constant and gi^ = J^gidx. Let us set C 2 := (l — ^||/i||oo)- We have 
C 2 > 0 because ||/z||oo < 1- Recall the fact that | > logt for any t > 0 which can be easily 
verified by showing that ije — logt is convex for t > 0 with minimum at e. Hence we have 

'> {^i^) 9i - -{^i • fi)Agi {1- -fi)Agi > C 2 {uJiA) gi 

e e 

which should be understood pointwisely. Hence, 

\\{^iA) giWh^ip) ^ 

Let C3 := \\{(jJiA)l\\Li(^Q) where l(x) = 1 for any x ^ Q. Using Ker(V) = {1/ G : u = 

c 1 a.e. for X G G M} together with the assumption Ker((x;^%l) f] Ker(V) = {0} one has 
C 3 > 0 . Using ( [T0| ) together with the fact that gi^ > 0 yields 

2 

\9in\ = \9in\ Cz = 

and thus ^ 

\9in\ < ^ ^ . E{gi). 

G2G3A 
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(B) Information 
loss + noise 



(C) Blur + noise 





(Bl) Method 0 


(B2) Method 


(B3) Method ^ 


(A4) Ours 



(B4) Ours 



Fig. 10. Three-phase vase segmentation (size: 481 x 321). (A): Given Gaussian noisy image with mean 0 and noise 
0.001; (B): Given Gaussian noisy image with 60% information loss; (C): Given blurry image with Gaussian noise; 
(A1-A4), (B1-B4) and (C1-C4): Results of methods | |44) , and our SLaT on (A), (B) and (C), respectively. 


Applying the triangular inequality in Q gives \\gi\\L^n) - \gin\ < CiW'^gih^a)- Hence 

ll5'i||L2(f2) < \gin\ + C*! II IIl2(Q) < 

Comparing with Q yet again shows that we have obtained 

\\9i\\w^^^{n) = \\9i\\L^{n) + \\'^9i\\L^{Q) < (^C2CsX ^ ^{9i)- 

Therefore, E is coercive. 

II) Uniqueness: Suppose gi^ and gi^ are both minimizers of E{gi). The convexity of E and the 
strict convexity of Agi {Agi — f log Agi) entail Agi^ = Agi^ on fig and Vgi^ = 'X/gi^. Further, 
the assumption on KeT{(jJiA) nKer(V) shows that gi^ = gi^. 

References 

[1] R Arbelaez, M. Maire, C. Fowlkes and J. Malik. Contour detection and hierarchical image segmentation. IEEE 
Transactions on Pattern Analysis and Machine Intelligence, 33(5):898-916, 2011. 

[2] H. Benninghoff and H. Garcke. Efficient Image Segmentation and Restoration Using Parametric Curve Evolution 
with Junctions and Topology Changes. SIAM Journal on Imaging Sciences, 7(3): 1451-1483, 2014. 






















16 






(B2) Method 


(C2) Method 



(B3) Method ^ (B4) Ours 



(C3) Method ^ 
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Fig. 11. Three-phase elephant segmentation (size: 321 x 481). (A): Given Poisson noisy image; (B): Given Poisson 
noisy image with 60% information loss; (C): Given blurry image with Poisson noise; (A1-A4), (B1-B4) and (C1-C4): 
Results of methods | [3T| , | [44) , and our SLaT on (A), (B) and (C), respectively. 
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Fig. 12. Four-phase man segmentation (size: 321 x 481). (A): Given Poisson noisy image; (B): Given Poisson noisy 
image with 60% information loss; (C): Given blurry image with Poisson noise; (A1-A4), (B1-B4) and (C1-C4): Results 
of methods | [3T| , | [44) , and our SLaT on (A), (B) and (C), respectively. 
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